On the correctness of monadic backward induction

In control theory, to solve a finite-horizon sequential decision problem (SDP) commonly means to find a list of decision rules that result in an optimal expected total reward (or cost) when taking a given number of decision steps. SDPs are routinely solved using Bellman\u27s backward induction. Textbook authors (e.g. Bertsekas or Puterman) typically give more or less formal proofs to show that the backward induction algorithm is correct as solution method for deterministic and stochastic SDPs. Botta, Jansson and Ionescu propose a generic framework for finite horizon, monadic SDPs together with a monadic version of backward induction for solving such SDPs. In monadic SDPs, the monad captures a generic notion of uncertainty, while a generic measure function aggregates rewards. In the present paper, we define a notion of correctness for monadic SDPs and identify three conditions that allow us to prove a correctness result for monadic backward induction that is comparable to textbook correctness proofs for ordinary backward induction. The conditions that we impose are fairly general and can be cast in category-theoretical terms using the notion of Eilenberg-Moore algebra. They hold in familiar settings like those of deterministic or stochastic SDPs, but we also give examples in which they fail. Our results show that backward induction can safely be employed for a broader class of SDPs than usually treated in textbooks. However, they also rule out certain instances that were considered admissible in the context of Botta et al. \u27s generic framework. Our development is formalised in Idris as an extension of the Botta et al. framework and the sources are available as supplementary material

On the logical structure of choice and bar induction principles

We develop an approach to choice principles and their contrapositive bar-induction principles as extensionality schemes connecting an "intensional" or "effective" view of respectively ill-and well-foundedness properties to an "extensional" or "ideal" view of these properties. After classifying and analysing the relations between different intensional definitions of ill-foundedness and well-foundedness, we introduce, for a domain $A$, a codomain $B$ and a "filter" $T$ on finite approximations of functions from $A$ to $B$, a generalised form GDC$_{A,B,T}$ of the axiom of dependent choice and dually a generalised bar induction principle GBI$_{A,B,T}$ such that: GDC$_{A,B,T}$ intuitionistically captures the strength of $\bullet$ the general axiom of choice expressed as $\forall a\exists b R(a, b) \Rightarrow\exists\alpha\forall \alpha R(\alpha,\alpha(a))$ when $T$ is a filter that derives point-wise from a relation $R$ on $A \times B$ without introducing further constraints, $\bullet$ the Boolean Prime Filter Theorem / Ultrafilter Theorem if $B$ is the two-element set $\mathbb{B}$ (for a constructive definition of prime filter), $\bullet$ the axiom of dependent choice if $A = \mathbb{N}$, $\bullet$ Weak K{\"o}nig's Lemma if $A = \mathbb{N}$ and $B = \mathbb{B}$ (up to weak classical reasoning) GBI$_{A,B,T}$ intuitionistically captures the strength of $\bullet$ G{\"o}del's completeness theorem in the form validity implies provability for entailment relations if $B = \mathbb{B}$, $\bullet$ bar induction when $A = \mathbb{N}$, $\bullet$ the Weak Fan Theorem when $A = \mathbb{N}$ and $B = \mathbb{B}$. Contrastingly, even though GDC$_{A,B,T}$ and GBI$_{A,B,T}$ smoothly capture several variants of choice and bar induction, some instances are inconsistent, e.g. when $A$ is $\mathbb{B}^\mathbb{N}$ and $B$ is $\mathbb{N}$.Comment: LICS 2021 - 36th Annual Symposium on Logic in Computer Science, Jun 2021, Rome / Virtual, Ital

Semantic verification of dynamic programming

We prove that the generic framework for specifying and solving finite-horizon, monadic sequential decision problems proposed in (Botta et al.,2017) is semantically correct. By semantically correct we mean that, for a problem specification $P$ and for any initial state $x$ compatible with $P$, the verified optimal policies obtained with the framework maximize the $P$-measure of the $P$-sums of the $P$-rewards along all the possible trajectories rooted in $x$. In short, we prove that, given $P$, the verified computations encoded in the framework are the correct computations to do. The main theorem is formulated as an equivalence between two value functions: the first lies at the core of dynamic programming as originally formulated in (Bellman,1957) and formalized by Botta et al. in Idris (Brady,2017), and the second is a specification. The equivalence requires the two value functions to be extensionally equal. Further, we identify and discuss three requirements that measures of uncertainty have to fulfill for the main theorem to hold. These turn out to be rather natural conditions that the expected-value measure of stochastic uncertainty fulfills. The formal proof of the main theorem crucially relies on a principle of preservation of extensional equality for functors. We formulate and prove the semantic correctness of dynamic programming as an extension of the Botta et al. Idris framework. However, the theory can easily be implemented in Coq or Agda.Comment: Manuscript ID: JFP-2020-003

On the logical structure of choice and bar induction principles

International audienceWe develop an approach to choice principles and their contrapositive bar-induction principles as extensionality schemes connecting an "intensional" or "effective" view of respectively ill-and well-foundedness properties to an "extensional" or "ideal" view of these properties. After classifying and analysing the relations between different intensional definitions of ill-foundedness and well-foundedness, we introduce, for a domain $A$, a codomain $B$ and a "filter" $T$ on finite approximations of functions from $A$ to $B$, a generalised form GDC$_{A,B,T}$ of the axiom of dependent choice and dually a generalised bar induction principle GBI$_{A,B,T}$ such that:- GDC$_{A,B,T}$ intuitionistically captures the strength of• the general axiom of choice expressed as $\forall a\exists\beta R(a, b) \Rightarrow\exists\alpha\forall a R(\alpha,(a \alpha (a)))$ when $T$ is a filter that derives point-wise from a relation $R$ on $A × B$ without introducing further constraints,• the Boolean Prime Filter Theorem / Ultrafilter Theorem if $B$ is the two-element set $\mathbb{B}$ (for a constructive definition of prime filter),• the axiom of dependent choice if $A = \mathbb{N}$,• Weak König’s Lemma if $A = \mathbb{N}$ and $B = \mathbb{B}$ (up to weak classical reasoning)- GBI$_{A,B,T}$ intuitionistically captures the strength of• Gödel’s completeness theorem in the form validity implies provability for entailment relations if $B = \mathbb{B}$,• bar induction when $A = \mathbb{N}$,• the Weak Fan Theorem when $A = \mathbb{N}$ and $B = \mathbb{B}$.Contrastingly, even though GDC$_{A,B,T}$ and GBI$_{A,B,T}$ smoothly capture several variants of choice and bar induction, some instances are inconsistent, e.g. when $A$ is $\mathbb{B}^\mathbb{N}$ and $B$ is $\mathbb{N}$

Extensional equality preservation and verified generic programming

In verified generic programming, one cannot exploit the structure of concrete data types but has to rely on well chosen sets of specifications or abstract data types (ADTs). Functors and monads are at the core of many applications of functional programming. This raises the question of what useful ADTs for verified functors and monads could look like. The functorial map of many important monads preserves extensional equality. For instance, if $f, g : A \rightarrow B$ are extensionally equal, that is, $\forall x \in A, \ f \ x = g \ x$, then $map \ f : List \ A \rightarrow List \ B$ and $map \ g$ are also extensionally equal. This suggests that preservation of extensional equality could be a useful principle in verified generic programming. We explore this possibility with a minimalist approach: we deal with (the lack of) extensional equality in Martin-L\"of's intensional type theories without extending the theories or using full-fledged setoids. Perhaps surprisingly, this minimal approach turns out to be extremely useful. It allows one to derive simple generic proofs of monadic laws but also verified, generic results in dynamical systems and control theory. In turn, these results avoid tedious code duplication and ad-hoc proofs. Thus, our work is a contribution towards pragmatic, verified generic programming.Comment: Manuscript ID: JFP-2020-003

SURFER v2.0: a flexible and simple model linking anthropogenic CO2 emissions and solar radiation modification to ocean acidification and sea level rise

We present SURFER, a novel reduced model for estimating the impact of CO2 emissions and solar radiation modification options on sea level rise and ocean acidification over timescales of several thousands of years. SURFER has been designed for the analysis of CO2 emission and solar radiation modification policies, for supporting the computation of optimal (CO2 emission and solar radiation modification) policies and for the study of commitment and responsibility under uncertainty. The model is based on a combination of conservation laws for the masses of atmospheric and oceanic carbon and for the oceanic temperature anomalies, and of ad-hoc parameterisations for the different sea level rise contributors: ice sheets, glaciers and ocean thermal expansion. It consists of 9 loosely coupled ordinary differential equations, is understandable, fast and easy to modify and calibrate. It reproduces the results of more sophisticated, high-dimensional earth system models on timescales up to millennia

SURFER v2.0: a flexible and simple model linking anthropogenic CO2 emissions and solar radiation modification to ocean acidification and sea level rise

We present SURFER, a novel reduced model for estimating the impact of CO2 emissions and solar radiation modification options on sea level rise and ocean acidification over timescales of several thousands of years. SURFER has been designed for the analysis of CO2 emission and solar radiation modification policies, for supporting the computation of optimal (CO2 emission and solar radiation modification) policies and for the study of commitment and responsibility under uncertainty. The model is based on a combination of conservation laws for the masses of atmospheric and oceanic carbon and for the oceanic temperature anomalies, and of adhoc parameterisations for the different sea level rise contributors: ice sheets, glaciers and ocean thermal expansion. It consists of 9 loosely coupled ordinary differential equations, is understandable, fast and easy to modify and calibrate. It reproduces the results of more sophisticated, high-dimensional earth system models on timescales up to millennia

Lost options commitment: how short-term policies affect long-term scope of action

We propose to explore the sustainability of climate policies based on a novel commitment metric. This metric allows to quantify how future generations' scope of action is affected by short-term climate policy. In an example application, we show that following a moderate emission scenario like SSP2-4.5 could commit future generations to heavily rely on carbon dioxide removal or/and solar radiation modification to avoid unmanageable sea level rise.Comment: 9 pages, 19 figure

Responsibility Under Uncertainty: Which Climate Decisions Matter Most?

We propose a new method for estimating how much decisions under monadic uncertainty matter. The method is generic and suitable for measuring responsibility in finite horizon sequential decision processes. It fulfills “fairness” requirements and three natural conditions for responsibility measures: agency, avoidance and causal relevance. We apply the method to study how much decisions matter in a stylized greenhouse gas emissions process in which a decision maker repeatedly faces two options: start a “green” transition to a decarbonized society or further delay such a transition. We account for the fact that climate decisions are rarely implemented with certainty and that their consequences on the climate and on the global economy are uncertain. We discover that a “moral” approach towards decision making — doing the right thing even though the probability of success becomes increasingly small — is rational over a wide range of uncertainties